from sklearn_benchmarks.reporting.hp_match import HPMatchReporting

scikit-learn vs. ONNX Runtime

reporting = HPMatchReporting("onnx", config="config.yml")
reporting.run()

KNeighborsClassifier_brute_force

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: algorithm=brute.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.047	0.018	0.0	0.017	-1	1	0.663	2.119	0.142	8.046	0.538	See	See
1	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.047	0.018	0.0	0.017	-1	1	0.663	0.029	0.006	593.657	123.748	See	See
2	KNeighborsClassifier_brute_force	predict	100000	1	100	0.313	0.008	0.0	0.313	-1	1	1.000	2.119	0.142	0.148	0.011	See	See
3	KNeighborsClassifier_brute_force	predict	100000	1	100	0.313	0.008	0.0	0.313	-1	1	1.000	0.029	0.006	10.900	2.290	See	See
4	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.029	0.022	0.0	0.017	-1	5	0.757	3.307	0.207	5.150	0.322	See	See
5	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.029	0.022	0.0	0.017	-1	5	0.757	0.028	0.003	608.080	74.062	See	See
6	KNeighborsClassifier_brute_force	predict	100000	1	100	0.311	0.006	0.0	0.311	-1	5	1.000	3.307	0.207	0.094	0.006	See	See
7	KNeighborsClassifier_brute_force	predict	100000	1	100	0.311	0.006	0.0	0.311	-1	5	1.000	0.028	0.003	11.116	1.372	See	See
8	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.116	0.010	0.0	0.017	1	100	0.882	2.160	0.034	7.924	0.126	See	See
9	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.116	0.010	0.0	0.017	1	100	0.882	0.025	0.000	674.334	8.475	See	See
10	KNeighborsClassifier_brute_force	predict	100000	1	100	0.314	0.006	0.0	0.314	1	100	1.000	2.160	0.034	0.145	0.004	See	See
11	KNeighborsClassifier_brute_force	predict	100000	1	100	0.314	0.006	0.0	0.314	1	100	1.000	0.025	0.000	12.365	0.280	See	See
12	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.291	0.015	0.0	0.017	-1	100	0.882	2.967	0.146	5.827	0.286	See	See
13	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.291	0.015	0.0	0.017	-1	100	0.882	0.028	0.004	609.215	78.278	See	See
14	KNeighborsClassifier_brute_force	predict	100000	1	100	0.312	0.007	0.0	0.312	-1	100	1.000	2.967	0.146	0.105	0.006	See	See
15	KNeighborsClassifier_brute_force	predict	100000	1	100	0.312	0.007	0.0	0.312	-1	100	1.000	0.028	0.004	11.008	1.437	See	See
16	KNeighborsClassifier_brute_force	predict	100000	1000	100	16.986	0.026	0.0	0.017	1	5	0.757	2.074	0.026	8.188	0.104	See	See
17	KNeighborsClassifier_brute_force	predict	100000	1000	100	16.986	0.026	0.0	0.017	1	5	0.757	0.024	0.000	715.448	8.001	See	See
18	KNeighborsClassifier_brute_force	predict	100000	1	100	0.311	0.007	0.0	0.311	1	5	1.000	2.074	0.026	0.150	0.004	See	See
19	KNeighborsClassifier_brute_force	predict	100000	1	100	0.311	0.007	0.0	0.311	1	5	1.000	0.024	0.000	13.119	0.331	See	See
20	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.538	0.032	0.0	0.018	1	1	0.663	1.198	0.018	14.644	0.218	See	See
21	KNeighborsClassifier_brute_force	predict	100000	1000	100	17.538	0.032	0.0	0.018	1	1	0.663	0.022	0.000	785.134	11.053	See	See
22	KNeighborsClassifier_brute_force	predict	100000	1	100	0.301	0.005	0.0	0.301	1	1	1.000	1.198	0.018	0.251	0.005	See	See
23	KNeighborsClassifier_brute_force	predict	100000	1	100	0.301	0.005	0.0	0.301	1	1	1.000	0.022	0.000	13.475	0.278	See	See
24	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.713	0.018	0.0	0.004	-1	1	0.896	1.736	0.028	2.139	0.037	See	See
25	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.713	0.018	0.0	0.004	-1	1	0.896	0.008	0.003	478.044	180.533	See	See
26	KNeighborsClassifier_brute_force	predict	100000	1	2	0.242	0.004	0.0	0.242	-1	1	1.000	1.736	0.028	0.139	0.003	See	See
27	KNeighborsClassifier_brute_force	predict	100000	1	2	0.242	0.004	0.0	0.242	-1	1	1.000	0.008	0.003	31.147	11.774	See	See
28	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.670	0.009	0.0	0.004	-1	5	0.922	2.645	0.025	1.387	0.013	See	See
29	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.670	0.009	0.0	0.004	-1	5	0.922	0.009	0.002	418.388	113.896	See	See
30	KNeighborsClassifier_brute_force	predict	100000	1	2	0.240	0.004	0.0	0.240	-1	5	1.000	2.645	0.025	0.091	0.002	See	See
31	KNeighborsClassifier_brute_force	predict	100000	1	2	0.240	0.004	0.0	0.240	-1	5	1.000	0.009	0.002	27.404	7.473	See	See
32	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.718	0.013	0.0	0.004	1	100	0.929	1.949	0.005	1.908	0.008	See	See
33	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.718	0.013	0.0	0.004	1	100	0.929	0.003	0.000	1352.009	81.064	See	See
34	KNeighborsClassifier_brute_force	predict	100000	1	2	0.241	0.004	0.0	0.241	1	100	1.000	1.949	0.005	0.124	0.002	See	See
35	KNeighborsClassifier_brute_force	predict	100000	1	2	0.241	0.004	0.0	0.241	1	100	1.000	0.003	0.000	87.607	5.475	See	See
36	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.729	0.053	0.0	0.004	-1	100	0.929	2.759	0.042	1.352	0.028	See	See
37	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.729	0.053	0.0	0.004	-1	100	0.929	0.011	0.006	328.972	161.590	See	See
38	KNeighborsClassifier_brute_force	predict	100000	1	2	0.242	0.004	0.0	0.242	-1	100	1.000	2.759	0.042	0.088	0.002	See	See
39	KNeighborsClassifier_brute_force	predict	100000	1	2	0.242	0.004	0.0	0.242	-1	100	1.000	0.011	0.006	21.353	10.489	See	See
40	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.680	0.011	0.0	0.004	1	5	0.922	2.029	0.023	1.814	0.021	See	See
41	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.680	0.011	0.0	0.004	1	5	0.922	0.003	0.000	1155.600	58.673	See	See
42	KNeighborsClassifier_brute_force	predict	100000	1	2	0.244	0.004	0.0	0.244	1	5	1.000	2.029	0.023	0.120	0.002	See	See
43	KNeighborsClassifier_brute_force	predict	100000	1	2	0.244	0.004	0.0	0.244	1	5	1.000	0.003	0.000	76.765	4.052	See	See
44	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.735	0.013	0.0	0.004	1	1	0.896	1.167	0.038	3.201	0.106	See	See
45	KNeighborsClassifier_brute_force	predict	100000	1000	2	3.735	0.013	0.0	0.004	1	1	0.896	0.002	0.000	2216.000	244.726	See	See
46	KNeighborsClassifier_brute_force	predict	100000	1	2	0.242	0.003	0.0	0.242	1	1	1.000	1.167	0.038	0.207	0.007	See	See
47	KNeighborsClassifier_brute_force	predict	100000	1	2	0.242	0.003	0.0	0.242	1	1	1.000	0.002	0.000	143.533	15.969	See	See

KNeighborsClassifier_kd_tree

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: algorithm=kd_tree.

	estimator	function	n_samples_train	n_samples	n_features	mean_duration	std_duration	iteration_throughput	latency	n_jobs	n_neighbors	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	122.937	0.000	0.0	0.123	-1	1	0.929	0.533	0.037	230.485	15.830	See	See
1	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	122.937	0.000	0.0	0.123	-1	1	0.929	0.002	0.000	49295.846	9103.424	See	See
2	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.635	0.078	0.0	2.635	-1	1	1.000	0.533	0.037	4.941	0.369	See	See
3	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.635	0.078	0.0	2.635	-1	1	1.000	0.002	0.000	1056.748	197.635	See	See
4	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	119.862	0.000	0.0	0.120	-1	5	0.946	1.147	0.034	104.512	3.089	See	See
5	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	119.862	0.000	0.0	0.120	-1	5	0.946	0.006	0.003	18562.390	8417.338	See	See
6	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.659	0.111	0.0	2.659	-1	5	1.000	1.147	0.034	2.319	0.118	See	See
7	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.659	0.111	0.0	2.659	-1	5	1.000	0.006	0.003	411.842	187.537	See	See
8	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	126.788	0.000	0.0	0.127	1	100	0.951	6.027	0.112	21.036	0.392	See	See
9	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	126.788	0.000	0.0	0.127	1	100	0.951	0.004	0.001	34572.115	10467.369	See	See
10	KNeighborsClassifier_kd_tree	predict	1000000	1	10	5.559	2.282	0.0	5.559	1	100	1.000	6.027	0.112	0.922	0.379	See	See
11	KNeighborsClassifier_kd_tree	predict	1000000	1	10	5.559	2.282	0.0	5.559	1	100	1.000	0.004	0.001	1515.946	773.160	See	See
12	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	125.777	0.000	0.0	0.126	-1	100	0.951	3.498	0.150	35.953	1.542	See	See
13	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	125.777	0.000	0.0	0.126	-1	100	0.951	0.006	0.001	20403.831	3282.812	See	See
14	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.661	0.148	0.0	2.661	-1	100	1.000	3.498	0.150	0.761	0.054	See	See
15	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.661	0.148	0.0	2.661	-1	100	1.000	0.006	0.001	431.655	73.504	See	See
16	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	136.202	0.000	0.0	0.136	1	5	0.946	1.822	0.042	74.764	1.741	See	See
17	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	136.202	0.000	0.0	0.136	1	5	0.946	0.002	0.001	87282.024	28977.488	See	See
18	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.665	0.123	0.0	2.665	1	5	1.000	1.822	0.042	1.463	0.076	See	See
19	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.665	0.123	0.0	2.665	1	5	1.000	0.002	0.001	1707.980	572.505	See	See
20	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	139.872	0.000	0.0	0.140	1	1	0.929	0.989	0.046	141.435	6.600	See	See
21	KNeighborsClassifier_kd_tree	predict	1000000	1000	10	139.872	0.000	0.0	0.140	1	1	0.929	0.001	0.000	138011.816	40488.770	See	See
22	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.635	0.064	0.0	2.635	1	1	1.000	0.989	0.046	2.664	0.140	See	See
23	KNeighborsClassifier_kd_tree	predict	1000000	1	10	2.635	0.064	0.0	2.635	1	1	1.000	0.001	0.000	2599.647	765.279	See	See
24	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.041	0.001	0.0	0.000	-1	1	0.891	0.022	0.001	1.925	0.088	See	See
25	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.041	0.001	0.0	0.000	-1	1	0.891	0.003	0.000	13.205	1.244	See	See
26	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	-1	1	1.000	0.022	0.001	0.270	0.013	See	See
27	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	-1	1	1.000	0.003	0.000	1.850	0.178	See	See
28	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.038	0.001	0.0	0.000	-1	5	0.911	0.025	0.002	1.515	0.111	See	See
29	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.038	0.001	0.0	0.000	-1	5	0.911	0.002	0.000	15.881	2.733	See	See
30	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	-1	5	1.000	0.025	0.002	0.229	0.016	See	See
31	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	-1	5	1.000	0.002	0.000	2.402	0.412	See	See
32	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.063	0.002	0.0	0.000	1	100	0.894	0.034	0.000	1.865	0.050	See	See
33	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.063	0.002	0.0	0.000	1	100	0.894	0.001	0.000	94.396	28.362	See	See
34	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	1	100	1.000	0.034	0.000	0.172	0.002	See	See
35	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	1	100	1.000	0.001	0.000	8.703	2.607	See	See
36	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.062	0.000	0.0	0.000	-1	100	0.894	0.037	0.003	1.672	0.142	See	See
37	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.062	0.000	0.0	0.000	-1	100	0.894	0.003	0.000	21.195	1.929	See	See
38	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	-1	100	1.000	0.037	0.003	0.154	0.013	See	See
39	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	-1	100	1.000	0.003	0.000	1.958	0.180	See	See
40	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.041	0.000	0.0	0.000	1	5	0.911	0.020	0.000	2.011	0.021	See	See
41	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.041	0.000	0.0	0.000	1	5	0.911	0.001	0.000	64.014	21.732	See	See
42	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	5	1.000	0.020	0.000	0.265	0.008	See	See
43	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.005	0.000	0.0	0.005	1	5	1.000	0.001	0.000	8.423	2.869	See	See
44	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.040	0.000	0.0	0.000	1	1	0.891	0.019	0.000	2.105	0.022	See	See
45	KNeighborsClassifier_kd_tree	predict	1000	1000	2	0.040	0.000	0.0	0.000	1	1	0.891	0.001	0.000	63.469	18.190	See	See
46	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	1	1	1.000	0.019	0.000	0.301	0.005	See	See
47	KNeighborsClassifier_kd_tree	predict	1000	1	2	0.006	0.000	0.0	0.006	1	1	1.000	0.001	0.000	9.063	2.601	See	See

HistGradientBoostingClassifier_best

onnx (1.10.1) vs. scikit-learn (1.0.dev0)

All estimators share the following hyperparameters: learning_rate=0.01, n_iter_no_change=10.0, max_leaf_nodes=100.0, max_bins=255.0, min_samples_leaf=100.0, max_iter=300.0.

	estimator	function	n_samples_train	n_samples	n_features	n_iter	mean_duration	std_duration	iteration_throughput	latency	accuracy_score	mean_duration_onnx	std_duration_onnx	speedup	std_speedup	sklearn_profiling	onnx_profiling
0	HistGradientBoostingClassifier_best	predict	100000	1000	100	300	0.486	0.036	0.002	0.000	0.824	0.107	0.001	4.560	0.343	See	See
1	HistGradientBoostingClassifier_best	predict	100000	1000	100	300	0.486	0.036	0.002	0.000	0.824	0.016	0.001	30.504	2.622	See	See
2	HistGradientBoostingClassifier_best	predict	100000	1	100	300	0.382	0.009	0.000	0.382	1.000	0.107	0.001	3.585	0.084	See	See
3	HistGradientBoostingClassifier_best	predict	100000	1	100	300	0.382	0.009	0.000	0.382	1.000	0.016	0.001	23.979	1.144	See	See